{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# 数据读取及预处理方法"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "3661e88cdd5db381"
  },
  {
   "cell_type": "markdown",
   "source": [
    "完整的数据分析流程：\n",
    "收集数据\n",
    "处理数据\n",
    "数据建模及可视化"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "ac091c72f55b5ffc"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 数据收集\n",
    "\n",
    "主要通过三种途径得到数据：\n",
    "1. 数据接口\n",
    "2. 数据库\n",
    "3. 数据爬虫"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "b5458953890b2902"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 数据读取"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "2e9c3b055e09afd8"
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "initial_id",
   "metadata": {
    "collapsed": true,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:54:47.491890200Z",
     "start_time": "2025-01-09T11:54:47.406961700Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File 'data04.zip' already there; not retrieving.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -nc \"http://labfile.oss.aliyuncs.com/courses/764/data04.zip\""
   ]
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Archive:  data04.zip\n",
      "  inflating: one_hot_demo.csv        \n",
      "  inflating: test_file.csv           \n",
      "  inflating: test_file.txt           \n",
      "  inflating: test_file.xlsx          \n",
      "  inflating: test_file_nan.csv       \n"
     ]
    }
   ],
   "source": [
    "!unzip -o \"data04.zip\" "
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:54:47.589113400Z",
     "start_time": "2025-01-09T11:54:47.483376600Z"
    }
   },
   "id": "43d46726e1093a30",
   "execution_count": 28
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 数据文件读取\n",
    ".csv .txt .xlsx\n",
    "open()方法"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "74fe3d7d95953564"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Zip Code,Total Population,Median Age,Total Males,Total Females,Total Households,Average Household Size\n",
      "\n",
      "91371,1,73.5,0,1,1,1\n",
      "\n",
      "90001,57110,26.6,28468,28642,12971,4.4\n",
      "\n",
      "90002,51223,25.5,24876,26347,11731,4.36\n",
      "\n",
      "90003,66266,26.3,32631,33635,15642,4.22\n",
      "\n",
      "90004,62180,34.8,31302,30878,22547,2.73\n",
      "\n",
      "90005,37681,33.9,19299,18382,15044,2.5\n",
      "\n",
      "90006,59185,32.4,30254,28931,18617,3.13\n",
      "\n",
      "90007,40920,24,20915,20005,11944,3\n",
      "\n",
      "90008,32327,39.7,14477,17850,13841,2.33\n",
      "\n",
      "90010,3800,37.8,1874,1926,2014,1.87\n",
      "\n",
      "90011,103892,26.2,52794,51098,22168,4.67\n",
      "\n",
      "90012,31103,36.3,19493,11610,10327,2.12\n",
      "\n",
      "90013,11772,44.6,7629,4143,6416,1.26\n",
      "\n",
      "90014,7005,44.8,4471,2534,4109,1.34\n",
      "\n",
      "90015,18986,31.3,9833,9153,7420,2.45\n",
      "\n",
      "90016,47596,33.9,22778,24818,16145,2.93\n",
      "\n",
      "90017,23768,29.4,12818,10950,9338,2.53\n",
      "\n",
      "90018,49310,33.2,23770,25540,15493,3.12\n",
      "\n",
      "90019,64458,35.8,31442,33016,23344,2.7\n",
      "\n",
      "90020,38967,34.6,19381,19586,16514,2.35\n",
      "\n",
      "90021,3951,44.3,2790,1161,1561,1.57\n",
      "\n",
      "90022,67179,29.8,33216,33963,17023,3.94\n",
      "\n",
      "90023,45903,28.4,23037,22866,10727,4.26\n",
      "\n",
      "90024,47452,23.6,22248,25204,17903,2.03\n",
      "\n",
      "90025,42147,34.7,20859,21288,21228,1.97\n",
      "\n",
      "90026,67869,34,34515,33354,24956,2.68\n",
      "\n",
      "90027,45151,38.3,22362,22789,21929,1.99\n",
      "\n",
      "90028,28714,34,16056,12658,14964,1.78\n",
      "\n",
      "90029,38617,34.6,19575,19042,13883,2.7\n",
      "\n",
      "90031,39316,33.5,19546,19770,11156,3.49\n",
      "\n",
      "90032,45786,32.4,22564,23222,12765,3.52\n",
      "\n",
      "90033,48852,29.2,24425,24427,12924,3.66\n",
      "\n",
      "90034,57964,32.8,28828,29136,25592,2.23\n",
      "\n",
      "90035,28418,37.5,13326,15092,12814,2.19\n",
      "\n",
      "90036,36865,33.9,17914,18951,18646,1.96\n",
      "\n",
      "90037,62276,28.8,31187,31089,15869,3.85\n",
      "\n",
      "90038,28917,33.1,15383,13534,11928,2.41\n",
      "\n",
      "90039,28514,38.8,14383,14131,11436,2.47\n",
      "\n",
      "90040,12520,31.2,6129,6391,3317,3.75\n",
      "\n",
      "90041,27425,39,13212,14213,9513,2.71\n",
      "\n",
      "90042,62430,33.6,30836,31594,19892,3.11\n",
      "\n",
      "90043,44789,38.7,20561,24228,16075,2.76\n",
      "\n",
      "90044,89779,28.6,43128,46651,25144,3.55\n",
      "\n",
      "90045,39480,35.6,18958,20522,15224,2.37\n",
      "\n",
      "90046,48581,38.5,25854,22727,28534,1.69\n",
      "\n",
      "90047,48606,36.2,22129,26477,16168,2.99\n",
      "\n",
      "90048,21397,39.2,10132,11265,11821,1.77\n",
      "\n",
      "90049,35482,41.3,16359,19123,16657,2.09\n",
      "\n",
      "90056,7827,48.4,3436,4391,3371,2.32\n",
      "\n",
      "90057,44998,31.2,24300,20698,15658,2.81\n",
      "\n",
      "90058,3223,26,1555,1668,892,3.6\n",
      "\n",
      "90059,40952,25.7,19623,21329,9596,4.19\n",
      "\n",
      "90061,26872,28.4,13049,13823,6892,3.85\n",
      "\n",
      "90062,32821,31.8,15720,17101,9155,3.55\n",
      "\n",
      "90063,55758,29,27843,27915,13260,4.19\n",
      "\n",
      "90064,25403,40,12297,13106,10968,2.29\n",
      "\n",
      "90065,45527,36.1,22873,22654,14476,3.12\n",
      "\n",
      "90066,55277,37.3,27714,27563,23985,2.29\n",
      "\n",
      "90067,2424,65.3,1074,1350,1510,1.61\n",
      "\n",
      "90068,22286,39.4,12018,10268,12326,1.8\n",
      "\n",
      "90069,20483,41.5,12153,8330,13364,1.53\n",
      "\n",
      "90071,15,45.5,13,2,0,0\n",
      "\n",
      "90073,539,56.9,506,33,4,1.25\n",
      "\n",
      "90077,9377,47.9,4594,4783,3615,2.57\n",
      "\n",
      "90079,0,0,0,0,0,0\n",
      "\n",
      "90089,3217,19.3,1436,1781,31,1.94\n",
      "\n",
      "90090,0,0,0,0,0,0\n",
      "\n",
      "90094,5464,33.7,2559,2905,2949,1.85\n",
      "\n",
      "90095,3,52.5,2,1,2,1.5\n",
      "\n",
      "90201,101279,27.8,50658,50621,24104,4.16\n",
      "\n",
      "90210,21741,47.5,10292,11449,8669,2.49\n",
      "\n",
      "90211,8434,40.6,3849,4585,3706,2.28\n",
      "\n",
      "90212,11555,41.2,5211,6344,5567,2.08\n",
      "\n",
      "90220,49328,29.8,23773,25555,12741,3.85\n",
      "\n",
      "90221,53704,26.7,26346,27358,11630,4.57\n",
      "\n",
      "90222,31869,27.3,15375,16494,7520,4.21\n",
      "\n",
      "90230,31766,39.1,14932,16834,12883,2.45\n",
      "\n",
      "90232,15149,38.6,7333,7816,6605,2.28\n",
      "\n",
      "90240,25876,35.5,12501,13375,7632,3.36\n",
      "\n",
      "90241,42399,33.9,20466,21933,13617,3.09\n",
      "\n",
      "90242,43497,31.6,21207,22290,12687,3.41\n",
      "\n",
      "90245,16654,39.2,8304,8350,7085,2.34\n",
      "\n",
      "90247,47487,35.5,23217,24270,15830,2.96\n",
      "\n",
      "90248,9947,41.2,4823,5124,3427,2.89\n",
      "\n",
      "90249,26669,37.2,12897,13772,8880,2.98\n",
      "\n",
      "90250,93193,31.9,45113,48080,31087,2.98\n",
      "\n",
      "90254,19506,37,10273,9233,9550,2.04\n",
      "\n",
      "90255,75066,29.1,37525,37541,18419,4.06\n",
      "\n",
      "90260,34924,32.7,17509,17415,10429,3.33\n",
      "\n",
      "90262,69745,27.8,33919,35826,14669,4.57\n",
      "\n",
      "90263,1612,19.7,665,947,0,0\n",
      "\n",
      "90265,18116,46.5,9159,8957,7174,2.39\n",
      "\n",
      "90266,35135,40.9,17605,17530,14038,2.5\n",
      "\n",
      "90270,27372,27.9,13992,13380,6554,4.16\n",
      "\n",
      "90272,22986,47.3,10952,12034,9212,2.49\n",
      "\n",
      "90274,25209,49.4,12199,13010,9479,2.66\n",
      "\n",
      "90275,41804,47.8,20283,21521,15618,2.65\n",
      "\n",
      "90277,35293,42,17521,17772,16910,2.07\n",
      "\n",
      "90278,40071,38.2,19848,20223,16009,2.49\n",
      "\n",
      "90280,94396,29.4,46321,48075,23278,4.05\n",
      "\n",
      "90290,6368,45,3180,3188,2612,2.44\n",
      "\n",
      "90291,28341,37.6,14757,13584,14261,1.95\n",
      "\n",
      "90292,21576,41.2,10729,10847,12654,1.7\n",
      "\n",
      "90293,12132,40.4,5890,6242,6575,1.83\n",
      "\n",
      "90301,36568,32.6,17633,18935,11895,3.01\n",
      "\n",
      "90302,29415,32.8,13803,15612,10684,2.74\n",
      "\n",
      "90303,26176,30.4,12733,13443,7290,3.58\n",
      "\n",
      "90304,28210,28.1,14503,13707,6634,4.22\n",
      "\n",
      "90305,14853,43.3,6552,8301,5933,2.46\n",
      "\n",
      "90401,6722,37.8,3524,3198,4188,1.49\n",
      "\n",
      "90402,12250,47,5809,6441,5301,2.31\n",
      "\n",
      "90403,24525,40.8,11426,13099,13970,1.74\n",
      "\n",
      "90404,21360,37.4,10292,11068,10089,2.01\n",
      "\n",
      "90405,27186,40.8,13364,13822,14376,1.87\n",
      "\n",
      "90501,43180,35,21483,21697,14610,2.94\n",
      "\n",
      "90502,18010,40.2,8811,9199,5717,2.94\n",
      "\n",
      "90503,44383,40.8,21481,22902,17183,2.55\n",
      "\n",
      "90504,32102,40.3,15886,16216,11580,2.76\n",
      "\n",
      "90505,36678,43,17684,18994,14244,2.55\n",
      "\n",
      "90506,0,0,0,0,0,0\n",
      "\n",
      "90601,31974,36.6,15312,16662,11027,2.89\n",
      "\n",
      "90602,25777,30.9,12720,13057,7980,3.09\n",
      "\n",
      "90603,20063,39.4,9694,10369,6788,2.93\n",
      "\n",
      "90604,39407,34.1,19270,20137,11932,3.29\n",
      "\n",
      "90605,40331,32.6,20033,20298,10527,3.81\n",
      "\n",
      "90606,32396,33.5,15936,16460,8633,3.72\n",
      "\n",
      "90621,35153,32.1,17377,17776,10304,3.38\n",
      "\n",
      "90623,15554,41.2,7516,8038,5072,3.06\n",
      "\n",
      "90630,47993,40,23204,24789,15785,3.02\n",
      "\n",
      "90631,67619,34.8,33320,34299,21452,3.13\n",
      "\n",
      "90638,49012,37.9,23520,25492,14821,3.11\n",
      "\n",
      "90640,62549,34.7,30189,32360,19027,3.27\n",
      "\n",
      "90650,105549,32.5,52364,53185,27130,3.83\n",
      "\n",
      "90660,62928,34,30738,32190,16564,3.77\n",
      "\n",
      "90670,14866,36.2,7163,7703,4393,3.34\n",
      "\n",
      "90701,16591,38.2,8227,8364,4553,3.51\n",
      "\n",
      "90703,49399,43.9,23785,25614,15604,3.16\n",
      "\n",
      "90704,4090,37.2,2101,1989,1629,2.5\n",
      "\n",
      "90706,76615,31.9,37203,39412,23650,3.21\n",
      "\n",
      "90710,25457,36.7,12291,13166,8717,2.91\n",
      "\n",
      "90712,31499,38.4,15277,16222,10794,2.91\n",
      "\n",
      "90713,27925,39,13574,14351,9447,2.95\n",
      "\n",
      "90715,20388,33.9,9935,10453,6105,3.33\n",
      "\n",
      "90716,14184,28.3,7078,7106,3515,4.03\n",
      "\n",
      "90717,21318,39.5,10317,11001,8520,2.48\n",
      "\n",
      "90720,21751,41.7,10423,11328,7789,2.74\n",
      "\n",
      "90723,54099,28.6,26315,27784,13882,3.87\n",
      "\n",
      "90731,59662,35.6,29951,29711,22044,2.62\n",
      "\n",
      "90732,21115,45,9843,11272,8611,2.38\n",
      "\n",
      "90740,23729,57.5,10423,13306,12830,1.83\n",
      "\n",
      "90744,53815,28.7,27298,26517,13999,3.83\n",
      "\n",
      "90745,57251,37.1,27754,29497,15210,3.73\n",
      "\n",
      "90746,25990,40.7,11944,14046,8050,3.14\n",
      "\n",
      "90747,0,0,0,0,0,0\n",
      "\n",
      "90755,11074,36.1,5462,5612,4172,2.64\n",
      "\n",
      "90802,39347,34.7,20387,18960,19853,1.93\n",
      "\n",
      "90803,32031,42.7,15609,16422,17318,1.84\n",
      "\n",
      "90804,40311,29.8,19686,20625,14556,2.7\n",
      "\n",
      "90805,93524,29,45229,48295,26056,3.56\n",
      "\n",
      "90806,42399,30.3,20717,21682,12184,3.44\n",
      "\n",
      "90807,31481,39.7,15153,16328,12452,2.49\n",
      "\n",
      "90808,38232,41.6,18408,19824,14167,2.69\n",
      "\n",
      "90810,36735,31.8,18067,18668,9289,3.9\n",
      "\n",
      "90813,58911,27.3,29425,29486,16425,3.5\n",
      "\n",
      "90814,19131,37.2,9356,9775,9170,2.06\n",
      "\n",
      "90815,39733,37.9,19124,20609,14836,2.53\n",
      "\n",
      "90822,117,63.9,109,8,2,4.5\n",
      "\n",
      "90831,0,0,0,0,0,0\n",
      "\n",
      "91001,36126,41.6,17421,18705,12663,2.83\n",
      "\n",
      "91006,31715,42.7,15283,16432,10652,2.98\n",
      "\n",
      "91007,34095,43.8,16039,18056,12033,2.73\n",
      "\n",
      "91008,1391,54.6,614,777,562,2.39\n",
      "\n",
      "91010,26074,38.1,12461,13613,7972,3.2\n",
      "\n",
      "91011,20280,45.9,9863,10417,6859,2.95\n",
      "\n",
      "91016,40598,37.9,19434,21164,15029,2.69\n",
      "\n",
      "91020,8415,40.2,3966,4449,3385,2.43\n",
      "\n",
      "91024,10917,46.6,5165,5752,4837,2.26\n",
      "\n",
      "91030,25616,40.1,12160,13456,10466,2.43\n",
      "\n",
      "91040,20372,43.3,10120,10252,7359,2.72\n",
      "\n",
      "91042,27585,40.7,13734,13851,9987,2.74\n",
      "\n",
      "91046,156,74,51,105,114,1.37\n",
      "\n",
      "91101,20460,33.7,9926,10534,10869,1.86\n",
      "\n",
      "91103,27480,35,13481,13999,8492,3.11\n",
      "\n",
      "91104,36751,38.2,17874,18877,12922,2.8\n",
      "\n",
      "91105,11254,49,5432,5822,5213,2.07\n",
      "\n",
      "91106,24229,34.6,11993,12236,10723,2.17\n",
      "\n",
      "91107,32940,41.2,15940,17000,13028,2.51\n",
      "\n",
      "91108,13361,45.4,6410,6951,4415,3.01\n",
      "\n",
      "91201,22781,40.4,11123,11658,8150,2.78\n",
      "\n",
      "91202,22830,41.7,10805,12025,8879,2.56\n",
      "\n",
      "91203,13220,38.9,6279,6941,5044,2.62\n",
      "\n",
      "91204,16032,38.1,7699,8333,5639,2.76\n",
      "\n",
      "91205,37810,39.1,18084,19726,14089,2.67\n",
      "\n",
      "91206,33065,42.6,15544,17521,13261,2.48\n",
      "\n",
      "91207,10506,45.5,4924,5582,4097,2.56\n",
      "\n",
      "91208,16245,43.6,7756,8489,6106,2.66\n",
      "\n",
      "91210,328,33.9,162,166,178,1.84\n",
      "\n",
      "91214,30356,42.5,14642,15714,10551,2.87\n",
      "\n",
      "91301,25488,42.7,12511,12977,9208,2.76\n",
      "\n",
      "91302,25709,42.4,12487,13222,9303,2.76\n",
      "\n",
      "91303,26855,31.1,13907,12948,8697,3.08\n",
      "\n",
      "91304,50231,35.9,24827,25404,16532,3\n",
      "\n",
      "91306,45061,35.4,22679,22382,13635,3.28\n",
      "\n",
      "91307,24474,43.4,11939,12535,8315,2.91\n",
      "\n",
      "91311,36557,42.7,17838,18719,13425,2.66\n",
      "\n",
      "91316,26898,42.1,12717,14181,11911,2.26\n",
      "\n",
      "91321,34882,33.3,17402,17480,11215,3.04\n",
      "\n",
      "91324,27669,36.5,13545,14124,9289,2.95\n",
      "\n",
      "91325,32417,35.4,15819,16598,11825,2.72\n",
      "\n",
      "91326,33708,42.7,16394,17314,11770,2.86\n",
      "\n",
      "91330,2702,19.6,1103,1599,2,2.5\n",
      "\n",
      "91331,103689,29.5,52358,51331,22465,4.6\n",
      "\n",
      "91335,74363,35.5,36596,37767,22855,3.21\n",
      "\n",
      "91340,34188,30.1,17130,17058,8176,4.17\n",
      "\n",
      "91342,91725,31.9,45786,45939,23543,3.83\n",
      "\n",
      "91343,60254,32.3,30145,30109,16802,3.55\n",
      "\n",
      "91344,51747,41,25212,26535,16872,3.03\n",
      "\n",
      "91345,18496,35.7,9110,9386,5192,3.52\n",
      "\n",
      "91350,33348,36.4,16270,17078,10991,3.01\n",
      "\n",
      "91351,32362,33.9,16086,16276,10036,3.22\n",
      "\n",
      "91352,47807,32.1,23980,23827,11985,3.95\n",
      "\n",
      "91354,28722,36.4,14046,14676,9522,3.01\n",
      "\n",
      "91355,32605,38.8,15773,16832,12772,2.51\n",
      "\n",
      "91356,29458,41.7,14216,15242,11686,2.5\n",
      "\n",
      "91361,20438,47.6,9854,10584,8374,2.43\n",
      "\n",
      "91362,36045,42.6,17562,18483,13563,2.65\n",
      "\n",
      "91364,25851,44,12739,13112,10409,2.47\n",
      "\n",
      "91367,39499,41,19171,20328,16697,2.35\n",
      "\n",
      "91377,13811,41.7,6641,7170,5158,2.68\n",
      "\n",
      "91381,20158,36,9852,10306,6636,3.04\n",
      "\n",
      "91384,29855,32.5,18503,11352,6925,3.3\n",
      "\n",
      "91387,40328,33,20106,20222,12871,3.13\n",
      "\n",
      "91390,19786,40.1,9911,9875,6369,3.07\n",
      "\n",
      "91401,39285,36,19613,19672,14380,2.7\n",
      "\n",
      "91402,69817,30.1,35095,34722,18194,3.81\n",
      "\n",
      "91403,23484,39.4,11277,12207,11358,2.06\n",
      "\n",
      "91405,51145,32.6,25498,25647,16256,3.11\n",
      "\n",
      "91406,51558,33.6,25956,25602,17042,3.02\n",
      "\n",
      "91411,24628,33.8,12495,12133,9177,2.63\n",
      "\n",
      "91423,30991,39.2,14789,16202,14876,2.07\n",
      "\n",
      "91436,14372,46.5,6890,7482,5552,2.57\n",
      "\n",
      "91501,20849,38.3,10040,10809,8235,2.53\n",
      "\n",
      "91502,11371,36.4,5402,5969,5001,2.26\n",
      "\n",
      "91504,24939,38.6,12006,12933,9180,2.69\n",
      "\n",
      "91505,30778,38.9,15121,15657,13180,2.33\n",
      "\n",
      "91506,18904,40.9,9115,9789,7555,2.45\n",
      "\n",
      "91601,37180,32.9,18932,18248,15970,2.32\n",
      "\n",
      "91602,17473,38,8722,8751,9277,1.88\n",
      "\n",
      "91604,29034,40.2,14271,14763,14292,2.01\n",
      "\n",
      "91605,56343,31.9,28417,27926,15357,3.61\n",
      "\n",
      "91606,44958,34.3,22376,22582,14903,3\n",
      "\n",
      "91607,27927,38.2,13635,14292,12859,2.15\n",
      "\n",
      "91608,0,0,0,0,0,0\n",
      "\n",
      "91702,59705,29.4,29486,30219,15455,3.67\n",
      "\n",
      "91706,76571,30.5,37969,38602,17504,4.35\n",
      "\n",
      "91709,74796,36.6,36954,37842,22940,3.25\n",
      "\n",
      "91710,80358,33,42283,38075,21952,3.43\n",
      "\n",
      "91711,35705,38.6,16777,18928,11868,2.58\n",
      "\n",
      "91722,34409,34,16859,17550,10079,3.41\n",
      "\n",
      "91723,18275,35,8783,9492,6296,2.86\n",
      "\n",
      "91724,26184,37.9,12780,13404,8508,3.05\n",
      "\n",
      "91731,29591,32.9,14888,14703,8007,3.66\n",
      "\n",
      "91732,61386,31.4,30755,30631,14700,4.13\n",
      "\n",
      "91733,43896,30.5,22191,21705,9918,4.4\n",
      "\n",
      "91740,25356,38.5,12269,13087,8376,2.94\n",
      "\n",
      "91741,25824,41.8,12498,13326,9126,2.82\n",
      "\n",
      "91744,85040,30.9,42564,42476,18648,4.55\n",
      "\n",
      "91745,54013,40.1,26287,27726,16188,3.33\n",
      "\n",
      "91746,30485,32.4,15116,15369,6743,4.5\n",
      "\n",
      "91748,45406,39.8,22368,23038,13311,3.4\n",
      "\n",
      "91750,33249,42.6,15881,17368,11944,2.71\n",
      "\n",
      "91754,32742,42.9,15620,17122,11193,2.92\n",
      "\n",
      "91755,27496,43.4,13271,14225,8760,3.12\n",
      "\n",
      "91759,476,47.2,239,237,216,2.2\n",
      "\n",
      "91763,36375,30.8,18099,18276,9450,3.81\n",
      "\n",
      "91765,46457,40.8,22702,23755,15039,3.08\n",
      "\n",
      "91766,71599,28.7,36111,35488,17708,4.01\n",
      "\n",
      "91767,48068,31.2,23685,24383,13691,3.47\n",
      "\n",
      "91768,34537,27.3,17509,17028,7885,3.87\n",
      "\n",
      "91770,62097,38.3,30521,31576,16588,3.7\n",
      "\n",
      "91773,33119,42.5,15737,17382,11941,2.73\n",
      "\n",
      "91775,23988,41.4,11448,12540,8227,2.9\n",
      "\n",
      "91776,38475,39,18751,19724,11776,3.24\n",
      "\n",
      "91780,34332,41.1,16438,17894,11318,3.03\n",
      "\n",
      "91784,25938,44.5,12729,13209,8934,2.9\n",
      "\n",
      "91786,51165,33,24516,26649,18087,2.79\n",
      "\n",
      "91789,43079,42.8,20988,22091,12891,3.31\n",
      "\n",
      "91790,44907,35.2,21721,23186,12751,3.49\n",
      "\n",
      "91791,32414,38,15512,16902,10236,3.15\n",
      "\n",
      "91792,30854,35.3,14950,15904,9154,3.35\n",
      "\n",
      "91801,52735,39.1,24833,27902,19315,2.71\n",
      "\n",
      "91803,30322,39.6,14486,15836,9894,3.04\n",
      "\n",
      "92301,32725,25.6,16857,15868,8132,3.81\n",
      "\n",
      "92371,16763,37.9,8565,8198,5370,3.11\n",
      "\n",
      "92372,6220,41.8,3136,3084,2198,2.83\n",
      "\n",
      "92397,4894,44.1,2522,2372,1998,2.45\n",
      "\n",
      "92821,35533,38.7,17338,18195,13062,2.72\n",
      "\n",
      "92823,3613,38.6,1757,1856,1154,3.13\n",
      "\n",
      "92833,51767,36,25643,26124,15849,3.23\n",
      "\n",
      "93040,2031,29.3,1052,979,522,3.89\n",
      "\n",
      "93063,54366,39,26592,27774,18650,2.9\n",
      "\n",
      "93225,5077,44,2585,2492,2080,2.44\n",
      "\n",
      "93243,1699,40.9,884,815,623,2.73\n",
      "\n",
      "93252,4176,38.4,3301,875,647,2.81\n",
      "\n",
      "93510,7993,45.2,4086,3907,2729,2.86\n",
      "\n",
      "93523,3074,27,1531,1543,1056,2.91\n",
      "\n",
      "93532,2932,41.7,1642,1290,1079,2.45\n",
      "\n",
      "93534,39341,31.1,18601,20740,14038,2.74\n",
      "\n",
      "93535,72046,28.3,34879,37167,20672,3.44\n",
      "\n",
      "93536,70918,34.4,37804,33114,20964,3.07\n",
      "\n",
      "93543,13033,32.9,6695,6338,3560,3.66\n",
      "\n",
      "93544,1259,52.4,689,570,569,2.2\n",
      "\n",
      "93550,74929,27.5,36414,38515,20864,3.58\n",
      "\n",
      "93551,50798,37,25056,25742,15963,3.18\n",
      "\n",
      "93552,38158,28.4,18711,19447,9690,3.93\n",
      "\n",
      "93553,2138,43.3,1121,1017,816,2.62\n",
      "\n",
      "93560,18910,32.4,9491,9419,6469,2.92\n",
      "\n",
      "93563,388,44.5,263,125,103,2.53\n",
      "\n",
      "93591,7285,30.9,3653,3632,1982,3.67\n"
     ]
    }
   ],
   "source": [
    "# 读取csv/txt文件\n",
    "with open(\"test_file.csv\",\"r\") as file:\n",
    "    for line in file:\n",
    "        print(line)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:54:47.590113400Z",
     "start_time": "2025-01-09T11:54:47.540112800Z"
    }
   },
   "id": "a9258a5ae26300f7",
   "execution_count": 29
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Requirement already satisfied: xlrd in d:\\0py\\scikit-learn\\.venv\\lib\\site-packages (2.0.1)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "[notice] A new release of pip is available: 23.2.1 -> 24.3.1\n",
      "[notice] To update, run: python.exe -m pip install --upgrade pip\n"
     ]
    }
   ],
   "source": [
    "!pip install xlrd"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:54:49.113060200Z",
     "start_time": "2025-01-09T11:54:47.545158900Z"
    }
   },
   "id": "284ef382cebc9c32",
   "execution_count": 30
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Requirement already satisfied: openpyxl in d:\\0py\\scikit-learn\\.venv\\lib\\site-packages (3.1.5)\n",
      "Requirement already satisfied: et-xmlfile in d:\\0py\\scikit-learn\\.venv\\lib\\site-packages (from openpyxl) (2.0.0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "[notice] A new release of pip is available: 23.2.1 -> 24.3.1\n",
      "[notice] To update, run: python.exe -m pip install --upgrade pip\n"
     ]
    }
   ],
   "source": [
    "!pip install openpyxl"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:54:50.488934800Z",
     "start_time": "2025-01-09T11:54:49.116060400Z"
    }
   },
   "id": "16b588d252f55e17",
   "execution_count": 31
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'load_workbook' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mNameError\u001B[0m                                 Traceback (most recent call last)",
      "Cell \u001B[1;32mIn[32], line 6\u001B[0m\n\u001B[0;32m      3\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mopenpyxl\u001B[39;00m\n\u001B[0;32m      5\u001B[0m \u001B[38;5;66;03m# 打开文件\u001B[39;00m\n\u001B[1;32m----> 6\u001B[0m file \u001B[38;5;241m=\u001B[39m \u001B[43mload_workbook\u001B[49m(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mtest_file.xlsx\u001B[39m\u001B[38;5;124m\"\u001B[39m)\n\u001B[0;32m      7\u001B[0m \u001B[38;5;66;03m# 按索引读取表\u001B[39;00m\n\u001B[0;32m      8\u001B[0m table \u001B[38;5;241m=\u001B[39m file\u001B[38;5;241m.\u001B[39msheet_by_index(\u001B[38;5;241m0\u001B[39m)\n",
      "\u001B[1;31mNameError\u001B[0m: name 'load_workbook' is not defined"
     ]
    }
   ],
   "source": [
    "# 对于 excel 支持的 .xlsx 文件就无法直接通过 open 读取了。需要 xlrd 包\n",
    "\n",
    "import openpyxl\n",
    "\n",
    "# 打开文件\n",
    "file = load_workbook(\"test_file.xlsx\")\n",
    "# 按索引读取表\n",
    "table = file.sheet_by_index(0)\n",
    "# 读取每行并打印\n",
    "for i in range(table.nrows):\n",
    "    print(table.row_values(i))"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:54:50.561958100Z",
     "start_time": "2025-01-09T11:54:50.489935400Z"
    }
   },
   "id": "10eb764fd40785b6",
   "execution_count": 32
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "     Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0       91371                 1        73.5            0              1   \n1       90001             57110        26.6        28468          28642   \n2       90002             51223        25.5        24876          26347   \n3       90003             66266        26.3        32631          33635   \n4       90004             62180        34.8        31302          30878   \n..        ...               ...         ...          ...            ...   \n314     93552             38158        28.4        18711          19447   \n315     93553              2138        43.3         1121           1017   \n316     93560             18910        32.4         9491           9419   \n317     93563               388        44.5          263            125   \n318     93591              7285        30.9         3653           3632   \n\n     Total Households  Average Household Size  \n0                   1                    1.00  \n1               12971                    4.40  \n2               11731                    4.36  \n3               15642                    4.22  \n4               22547                    2.73  \n..                ...                     ...  \n314              9690                    3.93  \n315               816                    2.62  \n316              6469                    2.92  \n317               103                    2.53  \n318              1982                    3.67  \n\n[319 rows x 7 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>91371</td>\n      <td>1</td>\n      <td>73.5</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>1.00</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>90001</td>\n      <td>57110</td>\n      <td>26.6</td>\n      <td>28468</td>\n      <td>28642</td>\n      <td>12971</td>\n      <td>4.40</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>90002</td>\n      <td>51223</td>\n      <td>25.5</td>\n      <td>24876</td>\n      <td>26347</td>\n      <td>11731</td>\n      <td>4.36</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>90003</td>\n      <td>66266</td>\n      <td>26.3</td>\n      <td>32631</td>\n      <td>33635</td>\n      <td>15642</td>\n      <td>4.22</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>90004</td>\n      <td>62180</td>\n      <td>34.8</td>\n      <td>31302</td>\n      <td>30878</td>\n      <td>22547</td>\n      <td>2.73</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>314</th>\n      <td>93552</td>\n      <td>38158</td>\n      <td>28.4</td>\n      <td>18711</td>\n      <td>19447</td>\n      <td>9690</td>\n      <td>3.93</td>\n    </tr>\n    <tr>\n      <th>315</th>\n      <td>93553</td>\n      <td>2138</td>\n      <td>43.3</td>\n      <td>1121</td>\n      <td>1017</td>\n      <td>816</td>\n      <td>2.62</td>\n    </tr>\n    <tr>\n      <th>316</th>\n      <td>93560</td>\n      <td>18910</td>\n      <td>32.4</td>\n      <td>9491</td>\n      <td>9419</td>\n      <td>6469</td>\n      <td>2.92</td>\n    </tr>\n    <tr>\n      <th>317</th>\n      <td>93563</td>\n      <td>388</td>\n      <td>44.5</td>\n      <td>263</td>\n      <td>125</td>\n      <td>103</td>\n      <td>2.53</td>\n    </tr>\n    <tr>\n      <th>318</th>\n      <td>93591</td>\n      <td>7285</td>\n      <td>30.9</td>\n      <td>3653</td>\n      <td>3632</td>\n      <td>1982</td>\n      <td>3.67</td>\n    </tr>\n  </tbody>\n</table>\n<p>319 rows × 7 columns</p>\n</div>"
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"test_file.csv\")\n",
    "df"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:01.244393700Z",
     "start_time": "2025-01-09T11:55:01.230057100Z"
    }
   },
   "id": "a1d5c327b2057383",
   "execution_count": 33
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "df1 = pd.read_excel(\"test_file.xlsx\")\n",
    "df1"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "start_time": "2025-01-09T11:54:50.548916100Z"
    }
   },
   "id": "45652d5f94dd485f",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "除了csv文件，Pandas 读取其他文件的方法如下：\n",
    "\n",
    "- pd.read_json # JSON 文件\n",
    "- pd.read_html # HTML 文件\n",
    "- pd.read_clipboard # 本地剪切板\n",
    "- pd.read_excel # MS Excel 文件\n",
    "- pd.read_hdf # HDF5Format 文件\n",
    "- pd.read_feather # Feather 格式\n",
    "- pd.read_msgpack # Msgpack\n",
    "- pd.read_stata # Stata\n",
    "- pd.read_sas # SAS \n",
    "- pd.read_pickle # Python Pickle 格式"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "61ecc45dceea9374"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# header=0 表示将第一行设为表头\n",
    "# sep='\\t'则代表使用空格分隔字段\n",
    "\n",
    "df2 = pd.read_csv(\"test_file.txt\", header=0, sep='\\t')\n",
    "df2"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "start_time": "2025-01-09T11:54:50.549916300Z"
    }
   },
   "id": "bee4f834dededf7a",
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 数据预处理"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "42cf7f3d4ef852e4"
  },
  {
   "cell_type": "markdown",
   "source": [
    "head() 和 tail() 方法可以帮助我们只预览数据集开头或结尾的一部分数据。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "54211e823da8700c"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "   Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0     91371                 1        73.5            0              1   \n1     90001             57110        26.6        28468          28642   \n2     90002             51223        25.5        24876          26347   \n3     90003             66266        26.3        32631          33635   \n4     90004             62180        34.8        31302          30878   \n\n   Total Households  Average Household Size  \n0                 1                    1.00  \n1             12971                    4.40  \n2             11731                    4.36  \n3             15642                    4.22  \n4             22547                    2.73  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>91371</td>\n      <td>1</td>\n      <td>73.5</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>1.00</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>90001</td>\n      <td>57110</td>\n      <td>26.6</td>\n      <td>28468</td>\n      <td>28642</td>\n      <td>12971</td>\n      <td>4.40</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>90002</td>\n      <td>51223</td>\n      <td>25.5</td>\n      <td>24876</td>\n      <td>26347</td>\n      <td>11731</td>\n      <td>4.36</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>90003</td>\n      <td>66266</td>\n      <td>26.3</td>\n      <td>32631</td>\n      <td>33635</td>\n      <td>15642</td>\n      <td>4.22</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>90004</td>\n      <td>62180</td>\n      <td>34.8</td>\n      <td>31302</td>\n      <td>30878</td>\n      <td>22547</td>\n      <td>2.73</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"test_file.csv\")\n",
    "df.head()  # 浏览头部数据"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:14.252561600Z",
     "start_time": "2025-01-09T11:55:14.236246Z"
    }
   },
   "id": "4e195af106ae8dd3",
   "execution_count": 34
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "     Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n314     93552             38158        28.4        18711          19447   \n315     93553              2138        43.3         1121           1017   \n316     93560             18910        32.4         9491           9419   \n317     93563               388        44.5          263            125   \n318     93591              7285        30.9         3653           3632   \n\n     Total Households  Average Household Size  \n314              9690                    3.93  \n315               816                    2.62  \n316              6469                    2.92  \n317               103                    2.53  \n318              1982                    3.67  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>314</th>\n      <td>93552</td>\n      <td>38158</td>\n      <td>28.4</td>\n      <td>18711</td>\n      <td>19447</td>\n      <td>9690</td>\n      <td>3.93</td>\n    </tr>\n    <tr>\n      <th>315</th>\n      <td>93553</td>\n      <td>2138</td>\n      <td>43.3</td>\n      <td>1121</td>\n      <td>1017</td>\n      <td>816</td>\n      <td>2.62</td>\n    </tr>\n    <tr>\n      <th>316</th>\n      <td>93560</td>\n      <td>18910</td>\n      <td>32.4</td>\n      <td>9491</td>\n      <td>9419</td>\n      <td>6469</td>\n      <td>2.92</td>\n    </tr>\n    <tr>\n      <th>317</th>\n      <td>93563</td>\n      <td>388</td>\n      <td>44.5</td>\n      <td>263</td>\n      <td>125</td>\n      <td>103</td>\n      <td>2.53</td>\n    </tr>\n    <tr>\n      <th>318</th>\n      <td>93591</td>\n      <td>7285</td>\n      <td>30.9</td>\n      <td>3653</td>\n      <td>3632</td>\n      <td>1982</td>\n      <td>3.67</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()  # 浏览尾部数据"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:15.863214900Z",
     "start_time": "2025-01-09T11:55:15.855168800Z"
    }
   },
   "id": "99b790eb8618d326",
   "execution_count": 35
  },
  {
   "cell_type": "markdown",
   "source": [
    "不带参数的head() 和 tail()方法默认显示 5 条数据，也可以自定义显示条数。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "e34c1b183fa17d0"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "   Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0     91371                 1        73.5            0              1   \n1     90001             57110        26.6        28468          28642   \n2     90002             51223        25.5        24876          26347   \n3     90003             66266        26.3        32631          33635   \n4     90004             62180        34.8        31302          30878   \n5     90005             37681        33.9        19299          18382   \n6     90006             59185        32.4        30254          28931   \n7     90007             40920        24.0        20915          20005   \n8     90008             32327        39.7        14477          17850   \n9     90010              3800        37.8         1874           1926   \n\n   Total Households  Average Household Size  \n0                 1                    1.00  \n1             12971                    4.40  \n2             11731                    4.36  \n3             15642                    4.22  \n4             22547                    2.73  \n5             15044                    2.50  \n6             18617                    3.13  \n7             11944                    3.00  \n8             13841                    2.33  \n9              2014                    1.87  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>91371</td>\n      <td>1</td>\n      <td>73.5</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>1.00</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>90001</td>\n      <td>57110</td>\n      <td>26.6</td>\n      <td>28468</td>\n      <td>28642</td>\n      <td>12971</td>\n      <td>4.40</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>90002</td>\n      <td>51223</td>\n      <td>25.5</td>\n      <td>24876</td>\n      <td>26347</td>\n      <td>11731</td>\n      <td>4.36</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>90003</td>\n      <td>66266</td>\n      <td>26.3</td>\n      <td>32631</td>\n      <td>33635</td>\n      <td>15642</td>\n      <td>4.22</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>90004</td>\n      <td>62180</td>\n      <td>34.8</td>\n      <td>31302</td>\n      <td>30878</td>\n      <td>22547</td>\n      <td>2.73</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>90005</td>\n      <td>37681</td>\n      <td>33.9</td>\n      <td>19299</td>\n      <td>18382</td>\n      <td>15044</td>\n      <td>2.50</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>90006</td>\n      <td>59185</td>\n      <td>32.4</td>\n      <td>30254</td>\n      <td>28931</td>\n      <td>18617</td>\n      <td>3.13</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>90007</td>\n      <td>40920</td>\n      <td>24.0</td>\n      <td>20915</td>\n      <td>20005</td>\n      <td>11944</td>\n      <td>3.00</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>90008</td>\n      <td>32327</td>\n      <td>39.7</td>\n      <td>14477</td>\n      <td>17850</td>\n      <td>13841</td>\n      <td>2.33</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>90010</td>\n      <td>3800</td>\n      <td>37.8</td>\n      <td>1874</td>\n      <td>1926</td>\n      <td>2014</td>\n      <td>1.87</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(10)  # 浏览头部 10 条数据"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:19.491932600Z",
     "start_time": "2025-01-09T11:55:19.474979700Z"
    }
   },
   "id": "9389066e5f7502d7",
   "execution_count": 36
  },
  {
   "cell_type": "markdown",
   "source": [
    "describe()方法可以对数据集中的数值进行统计，会输出数据计数、最大值、最小值等"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "32e2e68d64fadf5c"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "           Zip Code  Total Population  Median Age   Total Males  \\\ncount    319.000000        319.000000  319.000000    319.000000   \nmean   91000.673981      33241.341693   36.527586  16391.564263   \nstd      908.360203      21644.417455    8.692999  10747.495566   \nmin    90001.000000          0.000000    0.000000      0.000000   \n25%    90243.500000      19318.500000   32.400000   9763.500000   \n50%    90807.000000      31481.000000   37.100000  15283.000000   \n75%    91417.000000      44978.000000   41.000000  22219.500000   \nmax    93591.000000     105549.000000   74.000000  52794.000000   \n\n       Total Females  Total Households  Average Household Size  \ncount     319.000000        319.000000              319.000000  \nmean    16849.777429      10964.570533                2.828119  \nstd     10934.986468       6270.646400                0.835658  \nmin         0.000000          0.000000                0.000000  \n25%      9633.500000       6765.500000                2.435000  \n50%     16202.000000      10968.000000                2.830000  \n75%     22690.500000      14889.500000                3.320000  \nmax     53185.000000      31087.000000                4.670000  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>count</th>\n      <td>319.000000</td>\n      <td>319.000000</td>\n      <td>319.000000</td>\n      <td>319.000000</td>\n      <td>319.000000</td>\n      <td>319.000000</td>\n      <td>319.000000</td>\n    </tr>\n    <tr>\n      <th>mean</th>\n      <td>91000.673981</td>\n      <td>33241.341693</td>\n      <td>36.527586</td>\n      <td>16391.564263</td>\n      <td>16849.777429</td>\n      <td>10964.570533</td>\n      <td>2.828119</td>\n    </tr>\n    <tr>\n      <th>std</th>\n      <td>908.360203</td>\n      <td>21644.417455</td>\n      <td>8.692999</td>\n      <td>10747.495566</td>\n      <td>10934.986468</td>\n      <td>6270.646400</td>\n      <td>0.835658</td>\n    </tr>\n    <tr>\n      <th>min</th>\n      <td>90001.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n    </tr>\n    <tr>\n      <th>25%</th>\n      <td>90243.500000</td>\n      <td>19318.500000</td>\n      <td>32.400000</td>\n      <td>9763.500000</td>\n      <td>9633.500000</td>\n      <td>6765.500000</td>\n      <td>2.435000</td>\n    </tr>\n    <tr>\n      <th>50%</th>\n      <td>90807.000000</td>\n      <td>31481.000000</td>\n      <td>37.100000</td>\n      <td>15283.000000</td>\n      <td>16202.000000</td>\n      <td>10968.000000</td>\n      <td>2.830000</td>\n    </tr>\n    <tr>\n      <th>75%</th>\n      <td>91417.000000</td>\n      <td>44978.000000</td>\n      <td>41.000000</td>\n      <td>22219.500000</td>\n      <td>22690.500000</td>\n      <td>14889.500000</td>\n      <td>3.320000</td>\n    </tr>\n    <tr>\n      <th>max</th>\n      <td>93591.000000</td>\n      <td>105549.000000</td>\n      <td>74.000000</td>\n      <td>52794.000000</td>\n      <td>53185.000000</td>\n      <td>31087.000000</td>\n      <td>4.670000</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()  # 数据统计"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:26.960855400Z",
     "start_time": "2025-01-09T11:55:26.935543Z"
    }
   },
   "id": "2f17e4ad31924d0d",
   "execution_count": 37
  },
  {
   "cell_type": "markdown",
   "source": [
    "count() 方法可以用于计算数据集中非空数据的数量"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "9ac4a189f71d1457"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "Zip Code                  319\nTotal Population          319\nMedian Age                319\nTotal Males               319\nTotal Females             319\nTotal Households          319\nAverage Household Size    319\ndtype: int64"
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.count()  # 数据统计"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:28.639839900Z",
     "start_time": "2025-01-09T11:55:28.627017800Z"
    }
   },
   "id": "19b99614d2228139",
   "execution_count": 38
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 缺失值处理\n",
    "\n",
    "### 查找缺失值\n",
    "查找缺失值，我们依旧可以使用 Pandas 进行处理。\n",
    "Pandas 中，缺失数据一般采用 NaN 标记 NaN 代表 Not a Number。\n",
    "特别地，在时间序列里，时间戳的丢失采用 NaT 标记。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "89ca332301e3d37d"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "   Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0   91371.0               1.0        73.5          0.0            1.0   \n1   90001.0           57110.0        26.6      28468.0        28642.0   \n2   90002.0           51223.0        25.5      24876.0        26347.0   \n3       NaN           66266.0        26.3      32631.0        33635.0   \n4   90004.0           62180.0         NaN      31302.0        30878.0   \n5   90005.0           37681.0         NaN      19299.0        18382.0   \n6   90006.0           59185.0         NaN      30254.0        28931.0   \n7       NaN           40920.0         NaN      20915.0        20005.0   \n8   90008.0           32327.0         NaN      14477.0            NaN   \n9   90010.0            3800.0         NaN       1874.0            NaN   \n\n   Total Households  Average Household Size  \n0               1.0                    1.00  \n1           12971.0                    4.40  \n2           11731.0                    4.36  \n3           15642.0                    4.22  \n4           22547.0                    2.73  \n5           15044.0                    2.50  \n6           18617.0                    3.13  \n7           11944.0                    3.00  \n8           13841.0                    2.33  \n9            2014.0                    1.87  ",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>91371.0</td>\n      <td>1.0</td>\n      <td>73.5</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.00</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>90001.0</td>\n      <td>57110.0</td>\n      <td>26.6</td>\n      <td>28468.0</td>\n      <td>28642.0</td>\n      <td>12971.0</td>\n      <td>4.40</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>90002.0</td>\n      <td>51223.0</td>\n      <td>25.5</td>\n      <td>24876.0</td>\n      <td>26347.0</td>\n      <td>11731.0</td>\n      <td>4.36</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>NaN</td>\n      <td>66266.0</td>\n      <td>26.3</td>\n      <td>32631.0</td>\n      <td>33635.0</td>\n      <td>15642.0</td>\n      <td>4.22</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>90004.0</td>\n      <td>62180.0</td>\n      <td>NaN</td>\n      <td>31302.0</td>\n      <td>30878.0</td>\n      <td>22547.0</td>\n      <td>2.73</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>90005.0</td>\n      <td>37681.0</td>\n      <td>NaN</td>\n      <td>19299.0</td>\n      <td>18382.0</td>\n      <td>15044.0</td>\n      <td>2.50</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>90006.0</td>\n      <td>59185.0</td>\n      <td>NaN</td>\n      <td>30254.0</td>\n      <td>28931.0</td>\n      <td>18617.0</td>\n      <td>3.13</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>NaN</td>\n      <td>40920.0</td>\n      <td>NaN</td>\n      <td>20915.0</td>\n      <td>20005.0</td>\n      <td>11944.0</td>\n      <td>3.00</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>90008.0</td>\n      <td>32327.0</td>\n      <td>NaN</td>\n      <td>14477.0</td>\n      <td>NaN</td>\n      <td>13841.0</td>\n      <td>2.33</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>90010.0</td>\n      <td>3800.0</td>\n      <td>NaN</td>\n      <td>1874.0</td>\n      <td>NaN</td>\n      <td>2014.0</td>\n      <td>1.87</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"test_file_nan.csv\")\n",
    "df.head(10)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:31.590141700Z",
     "start_time": "2025-01-09T11:55:31.574214300Z"
    }
   },
   "id": "15dacdf280ef8fdd",
   "execution_count": 39
  },
  {
   "cell_type": "markdown",
   "source": [
    "pandas 用于检测缺失值主要用到两个方法，\n",
    "分别是：isnull() 和 notnull()，故名思意就是「是缺失值」和「不是缺失值」。\n",
    "默认会返回布尔值用于判断。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "a7a2fc0c34dfe38e"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "     Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0       False             False       False        False          False   \n1       False             False       False        False          False   \n2       False             False       False        False          False   \n3        True             False       False        False          False   \n4       False             False        True        False          False   \n..        ...               ...         ...          ...            ...   \n314     False             False       False        False          False   \n315     False             False       False        False          False   \n316     False             False       False        False          False   \n317     False             False       False        False          False   \n318     False             False       False        False          False   \n\n     Total Households  Average Household Size  \n0               False                   False  \n1               False                   False  \n2               False                   False  \n3               False                   False  \n4               False                   False  \n..                ...                     ...  \n314             False                   False  \n315             False                   False  \n316             False                   False  \n317             False                   False  \n318             False                   False  \n\n[319 rows x 7 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>False</td>\n      <td>False</td>\n      <td>True</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>314</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>315</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>316</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>317</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>318</th>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n      <td>False</td>\n    </tr>\n  </tbody>\n</table>\n<p>319 rows × 7 columns</p>\n</div>"
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:34.337749400Z",
     "start_time": "2025-01-09T11:55:34.325899900Z"
    }
   },
   "id": "240b64e6c76a20d9",
   "execution_count": 40
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "     Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0        True              True        True         True           True   \n1        True              True        True         True           True   \n2        True              True        True         True           True   \n3       False              True        True         True           True   \n4        True              True       False         True           True   \n..        ...               ...         ...          ...            ...   \n314      True              True        True         True           True   \n315      True              True        True         True           True   \n316      True              True        True         True           True   \n317      True              True        True         True           True   \n318      True              True        True         True           True   \n\n     Total Households  Average Household Size  \n0                True                    True  \n1                True                    True  \n2                True                    True  \n3                True                    True  \n4                True                    True  \n..                ...                     ...  \n314              True                    True  \n315              True                    True  \n316              True                    True  \n317              True                    True  \n318              True                    True  \n\n[319 rows x 7 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>False</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>True</td>\n      <td>True</td>\n      <td>False</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>314</th>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>315</th>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>316</th>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>317</th>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n    <tr>\n      <th>318</th>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n      <td>True</td>\n    </tr>\n  </tbody>\n</table>\n<p>319 rows × 7 columns</p>\n</div>"
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.notnull()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:36.589787100Z",
     "start_time": "2025-01-09T11:55:36.585267600Z"
    }
   },
   "id": "a5aaa04069ef5af",
   "execution_count": 41
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 删除缺失值\n",
    "\n",
    "适用于三种情况：\n",
    "\n",
    "第一，缺失值少，对数据集的影响可以忽略不计。\n",
    "缺失的数据行远远小于全部数据的数量，且删除这几行之后，对原数据集的影响可以忽略。\n",
    "*这时候，直接删除缺失值所在的行是最好的。*\n",
    "\n",
    "第二，缺失数据量大。\n",
    "举个例子，一个数据集有 1 万行，存在 10 个特征列。\n",
    "其中某一项特征所在的列存在 9000 个空值。\n",
    "这也就表明该列存在的意义已经不大了。\n",
    "所以也需要删除数据。\n",
    "\n",
    "第三，该缺失值无法被填充。\n",
    "可能有实际意义的特征。\n",
    "所以，对应这样的数据行已经没有意义，选择直接删除往往是最好的。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "45a1106bbaad7a98"
  },
  {
   "cell_type": "markdown",
   "source": [
    "删除缺失值所在的列或者行非常简单。\n",
    "使用pandas提供的dropna()方法。\n",
    "dropna() 方法可以将有缺失值的行或列全部移除。\n",
    "当然，你可以使用 axis=0 参数指定行，或 axis=1 参数指定列。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "b46df00cb196b286"
  },
  {
   "cell_type": "markdown",
   "source": [
    "删除有缺失值列之后，只剩下索引了。因为原数据集每一列均有空值。所以，删除列操作要慎用。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "25fa44af4526009f"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "     Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0     91371.0               1.0        73.5          0.0            1.0   \n1     90001.0           57110.0        26.6      28468.0        28642.0   \n2     90002.0           51223.0        25.5      24876.0        26347.0   \n16    90017.0           23768.0        29.4      12818.0        10950.0   \n17    90018.0           49310.0        33.2      23770.0        25540.0   \n..        ...               ...         ...          ...            ...   \n314   93552.0           38158.0        28.4      18711.0        19447.0   \n315   93553.0            2138.0        43.3       1121.0         1017.0   \n316   93560.0           18910.0        32.4       9491.0         9419.0   \n317   93563.0             388.0        44.5        263.0          125.0   \n318   93591.0            7285.0        30.9       3653.0         3632.0   \n\n     Total Households  Average Household Size  \n0                 1.0                    1.00  \n1             12971.0                    4.40  \n2             11731.0                    4.36  \n16             9338.0                    2.53  \n17            15493.0                    3.12  \n..                ...                     ...  \n314            9690.0                    3.93  \n315             816.0                    2.62  \n316            6469.0                    2.92  \n317             103.0                    2.53  \n318            1982.0                    3.67  \n\n[291 rows x 7 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>91371.0</td>\n      <td>1.0</td>\n      <td>73.5</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.00</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>90001.0</td>\n      <td>57110.0</td>\n      <td>26.6</td>\n      <td>28468.0</td>\n      <td>28642.0</td>\n      <td>12971.0</td>\n      <td>4.40</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>90002.0</td>\n      <td>51223.0</td>\n      <td>25.5</td>\n      <td>24876.0</td>\n      <td>26347.0</td>\n      <td>11731.0</td>\n      <td>4.36</td>\n    </tr>\n    <tr>\n      <th>16</th>\n      <td>90017.0</td>\n      <td>23768.0</td>\n      <td>29.4</td>\n      <td>12818.0</td>\n      <td>10950.0</td>\n      <td>9338.0</td>\n      <td>2.53</td>\n    </tr>\n    <tr>\n      <th>17</th>\n      <td>90018.0</td>\n      <td>49310.0</td>\n      <td>33.2</td>\n      <td>23770.0</td>\n      <td>25540.0</td>\n      <td>15493.0</td>\n      <td>3.12</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>314</th>\n      <td>93552.0</td>\n      <td>38158.0</td>\n      <td>28.4</td>\n      <td>18711.0</td>\n      <td>19447.0</td>\n      <td>9690.0</td>\n      <td>3.93</td>\n    </tr>\n    <tr>\n      <th>315</th>\n      <td>93553.0</td>\n      <td>2138.0</td>\n      <td>43.3</td>\n      <td>1121.0</td>\n      <td>1017.0</td>\n      <td>816.0</td>\n      <td>2.62</td>\n    </tr>\n    <tr>\n      <th>316</th>\n      <td>93560.0</td>\n      <td>18910.0</td>\n      <td>32.4</td>\n      <td>9491.0</td>\n      <td>9419.0</td>\n      <td>6469.0</td>\n      <td>2.92</td>\n    </tr>\n    <tr>\n      <th>317</th>\n      <td>93563.0</td>\n      <td>388.0</td>\n      <td>44.5</td>\n      <td>263.0</td>\n      <td>125.0</td>\n      <td>103.0</td>\n      <td>2.53</td>\n    </tr>\n    <tr>\n      <th>318</th>\n      <td>93591.0</td>\n      <td>7285.0</td>\n      <td>30.9</td>\n      <td>3653.0</td>\n      <td>3632.0</td>\n      <td>1982.0</td>\n      <td>3.67</td>\n    </tr>\n  </tbody>\n</table>\n<p>291 rows × 7 columns</p>\n</div>"
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 原数据集有 319 行，7 列。删除含有缺失值的行之后，还剩下 291 行，7 列\n",
    "df.dropna(axis=0)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:42.027177800Z",
     "start_time": "2025-01-09T11:55:42.014797800Z"
    }
   },
   "id": "1414f756396a7b05",
   "execution_count": 42
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "Empty DataFrame\nColumns: []\nIndex: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, ...]\n\n[319 rows x 0 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n    </tr>\n    <tr>\n      <th>1</th>\n    </tr>\n    <tr>\n      <th>2</th>\n    </tr>\n    <tr>\n      <th>3</th>\n    </tr>\n    <tr>\n      <th>4</th>\n    </tr>\n    <tr>\n      <th>...</th>\n    </tr>\n    <tr>\n      <th>314</th>\n    </tr>\n    <tr>\n      <th>315</th>\n    </tr>\n    <tr>\n      <th>316</th>\n    </tr>\n    <tr>\n      <th>317</th>\n    </tr>\n    <tr>\n      <th>318</th>\n    </tr>\n  </tbody>\n</table>\n<p>319 rows × 0 columns</p>\n</div>"
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dropna(axis=1)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:55:44.108783700Z",
     "start_time": "2025-01-09T11:55:44.097447500Z"
    }
   },
   "id": "40f4d7ef67502f06",
   "execution_count": 43
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 填充缺失值\n",
    "\n",
    "一般情况下，填充缺失值有三种方法。\n",
    "\n",
    "第一，手动填充。\n",
    "\n",
    "第二，临近填充。\n",
    "故名思意就是采用与缺失值相邻的数据进行填充缺失值的方法。\n",
    "临近填充比较适合于零散的不确定数据。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "d75e7f58b80e4ac1"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\bzj27\\AppData\\Local\\Temp\\ipykernel_81960\\1485656668.py:2: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  df.fillna(method='pad')\n"
     ]
    },
    {
     "data": {
      "text/plain": "     Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0     91371.0               1.0        73.5          0.0            1.0   \n1     90001.0           57110.0        26.6      28468.0        28642.0   \n2     90002.0           51223.0        25.5      24876.0        26347.0   \n3     90002.0           66266.0        26.3      32631.0        33635.0   \n4     90004.0           62180.0        26.3      31302.0        30878.0   \n..        ...               ...         ...          ...            ...   \n314   93552.0           38158.0        28.4      18711.0        19447.0   \n315   93553.0            2138.0        43.3       1121.0         1017.0   \n316   93560.0           18910.0        32.4       9491.0         9419.0   \n317   93563.0             388.0        44.5        263.0          125.0   \n318   93591.0            7285.0        30.9       3653.0         3632.0   \n\n     Total Households  Average Household Size  \n0                 1.0                    1.00  \n1             12971.0                    4.40  \n2             11731.0                    4.36  \n3             15642.0                    4.22  \n4             22547.0                    2.73  \n..                ...                     ...  \n314            9690.0                    3.93  \n315             816.0                    2.62  \n316            6469.0                    2.92  \n317             103.0                    2.53  \n318            1982.0                    3.67  \n\n[319 rows x 7 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>91371.0</td>\n      <td>1.0</td>\n      <td>73.5</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.00</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>90001.0</td>\n      <td>57110.0</td>\n      <td>26.6</td>\n      <td>28468.0</td>\n      <td>28642.0</td>\n      <td>12971.0</td>\n      <td>4.40</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>90002.0</td>\n      <td>51223.0</td>\n      <td>25.5</td>\n      <td>24876.0</td>\n      <td>26347.0</td>\n      <td>11731.0</td>\n      <td>4.36</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>90002.0</td>\n      <td>66266.0</td>\n      <td>26.3</td>\n      <td>32631.0</td>\n      <td>33635.0</td>\n      <td>15642.0</td>\n      <td>4.22</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>90004.0</td>\n      <td>62180.0</td>\n      <td>26.3</td>\n      <td>31302.0</td>\n      <td>30878.0</td>\n      <td>22547.0</td>\n      <td>2.73</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>314</th>\n      <td>93552.0</td>\n      <td>38158.0</td>\n      <td>28.4</td>\n      <td>18711.0</td>\n      <td>19447.0</td>\n      <td>9690.0</td>\n      <td>3.93</td>\n    </tr>\n    <tr>\n      <th>315</th>\n      <td>93553.0</td>\n      <td>2138.0</td>\n      <td>43.3</td>\n      <td>1121.0</td>\n      <td>1017.0</td>\n      <td>816.0</td>\n      <td>2.62</td>\n    </tr>\n    <tr>\n      <th>316</th>\n      <td>93560.0</td>\n      <td>18910.0</td>\n      <td>32.4</td>\n      <td>9491.0</td>\n      <td>9419.0</td>\n      <td>6469.0</td>\n      <td>2.92</td>\n    </tr>\n    <tr>\n      <th>317</th>\n      <td>93563.0</td>\n      <td>388.0</td>\n      <td>44.5</td>\n      <td>263.0</td>\n      <td>125.0</td>\n      <td>103.0</td>\n      <td>2.53</td>\n    </tr>\n    <tr>\n      <th>318</th>\n      <td>93591.0</td>\n      <td>7285.0</td>\n      <td>30.9</td>\n      <td>3653.0</td>\n      <td>3632.0</td>\n      <td>1982.0</td>\n      <td>3.67</td>\n    </tr>\n  </tbody>\n</table>\n<p>319 rows × 7 columns</p>\n</div>"
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pandas 提供了用于临近填充的 fillna() 方法。\n",
    "# 被前面的临近值进行了填充\n",
    "\n",
    "df.fillna(method='pad')"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:56:19.550745600Z",
     "start_time": "2025-01-09T11:56:19.518860500Z"
    }
   },
   "id": "1380b04523c74452",
   "execution_count": 44
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\bzj27\\AppData\\Local\\Temp\\ipykernel_81960\\4194935721.py:1: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.\n",
      "  df.fillna(method='bfill')\n"
     ]
    },
    {
     "data": {
      "text/plain": "     Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0     91371.0               1.0        73.5          0.0            1.0   \n1     90001.0           57110.0        26.6      28468.0        28642.0   \n2     90002.0           51223.0        25.5      24876.0        26347.0   \n3     90004.0           66266.0        26.3      32631.0        33635.0   \n4     90004.0           62180.0        26.2      31302.0        30878.0   \n..        ...               ...         ...          ...            ...   \n314   93552.0           38158.0        28.4      18711.0        19447.0   \n315   93553.0            2138.0        43.3       1121.0         1017.0   \n316   93560.0           18910.0        32.4       9491.0         9419.0   \n317   93563.0             388.0        44.5        263.0          125.0   \n318   93591.0            7285.0        30.9       3653.0         3632.0   \n\n     Total Households  Average Household Size  \n0                 1.0                    1.00  \n1             12971.0                    4.40  \n2             11731.0                    4.36  \n3             15642.0                    4.22  \n4             22547.0                    2.73  \n..                ...                     ...  \n314            9690.0                    3.93  \n315             816.0                    2.62  \n316            6469.0                    2.92  \n317             103.0                    2.53  \n318            1982.0                    3.67  \n\n[319 rows x 7 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>91371.0</td>\n      <td>1.0</td>\n      <td>73.5</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.00</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>90001.0</td>\n      <td>57110.0</td>\n      <td>26.6</td>\n      <td>28468.0</td>\n      <td>28642.0</td>\n      <td>12971.0</td>\n      <td>4.40</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>90002.0</td>\n      <td>51223.0</td>\n      <td>25.5</td>\n      <td>24876.0</td>\n      <td>26347.0</td>\n      <td>11731.0</td>\n      <td>4.36</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>90004.0</td>\n      <td>66266.0</td>\n      <td>26.3</td>\n      <td>32631.0</td>\n      <td>33635.0</td>\n      <td>15642.0</td>\n      <td>4.22</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>90004.0</td>\n      <td>62180.0</td>\n      <td>26.2</td>\n      <td>31302.0</td>\n      <td>30878.0</td>\n      <td>22547.0</td>\n      <td>2.73</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>314</th>\n      <td>93552.0</td>\n      <td>38158.0</td>\n      <td>28.4</td>\n      <td>18711.0</td>\n      <td>19447.0</td>\n      <td>9690.0</td>\n      <td>3.93</td>\n    </tr>\n    <tr>\n      <th>315</th>\n      <td>93553.0</td>\n      <td>2138.0</td>\n      <td>43.3</td>\n      <td>1121.0</td>\n      <td>1017.0</td>\n      <td>816.0</td>\n      <td>2.62</td>\n    </tr>\n    <tr>\n      <th>316</th>\n      <td>93560.0</td>\n      <td>18910.0</td>\n      <td>32.4</td>\n      <td>9491.0</td>\n      <td>9419.0</td>\n      <td>6469.0</td>\n      <td>2.92</td>\n    </tr>\n    <tr>\n      <th>317</th>\n      <td>93563.0</td>\n      <td>388.0</td>\n      <td>44.5</td>\n      <td>263.0</td>\n      <td>125.0</td>\n      <td>103.0</td>\n      <td>2.53</td>\n    </tr>\n    <tr>\n      <th>318</th>\n      <td>93591.0</td>\n      <td>7285.0</td>\n      <td>30.9</td>\n      <td>3653.0</td>\n      <td>3632.0</td>\n      <td>1982.0</td>\n      <td>3.67</td>\n    </tr>\n  </tbody>\n</table>\n<p>319 rows × 7 columns</p>\n</div>"
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.fillna(method='bfill')   # 使用后面的临近值进行填充。"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:56:34.996676700Z",
     "start_time": "2025-01-09T11:56:34.981132900Z"
    }
   },
   "id": "563a0ad6d7cf75f0",
   "execution_count": 45
  },
  {
   "cell_type": "markdown",
   "source": [
    "第三，插值填充。\n",
    "插值填充就是采用数学的方法对数据进行插值。\n",
    "举个例子，有一列数据为 [2011, 2012, 2013, NaN，NaN，2016, 2017]。\n",
    "这里，无论你采用向前还是向后填充，其实都不是最好的。\n",
    "你可以发现数据是一个等差数列，\n",
    "缺失值应该分别为[2014, 2015]，这也就是一个线性插值的过程。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "304b3f82afcefacd"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "     Zip Code  Total Population  Median Age  Total Males  Total Females  \\\n0     91371.0               1.0   73.500000          0.0            1.0   \n1     90001.0           57110.0   26.600000      28468.0        28642.0   \n2     90002.0           51223.0   25.500000      24876.0        26347.0   \n3     90003.0           66266.0   26.300000      32631.0        33635.0   \n4     90004.0           62180.0   26.285714      31302.0        30878.0   \n..        ...               ...         ...          ...            ...   \n314   93552.0           38158.0   28.400000      18711.0        19447.0   \n315   93553.0            2138.0   43.300000       1121.0         1017.0   \n316   93560.0           18910.0   32.400000       9491.0         9419.0   \n317   93563.0             388.0   44.500000        263.0          125.0   \n318   93591.0            7285.0   30.900000       3653.0         3632.0   \n\n     Total Households  Average Household Size  \n0                 1.0                    1.00  \n1             12971.0                    4.40  \n2             11731.0                    4.36  \n3             15642.0                    4.22  \n4             22547.0                    2.73  \n..                ...                     ...  \n314            9690.0                    3.93  \n315             816.0                    2.62  \n316            6469.0                    2.92  \n317             103.0                    2.53  \n318            1982.0                    3.67  \n\n[319 rows x 7 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Zip Code</th>\n      <th>Total Population</th>\n      <th>Median Age</th>\n      <th>Total Males</th>\n      <th>Total Females</th>\n      <th>Total Households</th>\n      <th>Average Household Size</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>91371.0</td>\n      <td>1.0</td>\n      <td>73.500000</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>1.0</td>\n      <td>1.00</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>90001.0</td>\n      <td>57110.0</td>\n      <td>26.600000</td>\n      <td>28468.0</td>\n      <td>28642.0</td>\n      <td>12971.0</td>\n      <td>4.40</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>90002.0</td>\n      <td>51223.0</td>\n      <td>25.500000</td>\n      <td>24876.0</td>\n      <td>26347.0</td>\n      <td>11731.0</td>\n      <td>4.36</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>90003.0</td>\n      <td>66266.0</td>\n      <td>26.300000</td>\n      <td>32631.0</td>\n      <td>33635.0</td>\n      <td>15642.0</td>\n      <td>4.22</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>90004.0</td>\n      <td>62180.0</td>\n      <td>26.285714</td>\n      <td>31302.0</td>\n      <td>30878.0</td>\n      <td>22547.0</td>\n      <td>2.73</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>314</th>\n      <td>93552.0</td>\n      <td>38158.0</td>\n      <td>28.400000</td>\n      <td>18711.0</td>\n      <td>19447.0</td>\n      <td>9690.0</td>\n      <td>3.93</td>\n    </tr>\n    <tr>\n      <th>315</th>\n      <td>93553.0</td>\n      <td>2138.0</td>\n      <td>43.300000</td>\n      <td>1121.0</td>\n      <td>1017.0</td>\n      <td>816.0</td>\n      <td>2.62</td>\n    </tr>\n    <tr>\n      <th>316</th>\n      <td>93560.0</td>\n      <td>18910.0</td>\n      <td>32.400000</td>\n      <td>9491.0</td>\n      <td>9419.0</td>\n      <td>6469.0</td>\n      <td>2.92</td>\n    </tr>\n    <tr>\n      <th>317</th>\n      <td>93563.0</td>\n      <td>388.0</td>\n      <td>44.500000</td>\n      <td>263.0</td>\n      <td>125.0</td>\n      <td>103.0</td>\n      <td>2.53</td>\n    </tr>\n    <tr>\n      <th>318</th>\n      <td>93591.0</td>\n      <td>7285.0</td>\n      <td>30.900000</td>\n      <td>3653.0</td>\n      <td>3632.0</td>\n      <td>1982.0</td>\n      <td>3.67</td>\n    </tr>\n  </tbody>\n</table>\n<p>319 rows × 7 columns</p>\n</div>"
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 通过 interpolate() 方法实现\n",
    "# 默认为参数为线性插值，即 method='linear'\n",
    "\n",
    "df.interpolate()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2025-01-09T11:58:16.725480900Z",
     "start_time": "2025-01-09T11:58:16.706763800Z"
    }
   },
   "id": "7293fef461cbd4bb",
   "execution_count": 46
  },
  {
   "cell_type": "markdown",
   "source": [
    "除此之外，interpolate()方法还有 linear, time, index, values, nearest, zero, slinear, quadratic, cubic, barycentric, krogh, polynomial, spline, piecewise_polynomial, from_derivatives, pchip, akima 等插值方法可供选择。\n",
    "\n",
    "- 如果你的数据增长速率越来越快，可以选择 method='quadratic' 二次插值。\n",
    "- 如果数据集呈现出累计分布的样子，推荐选择 method='pchip'。\n",
    "- 如果需要填补缺省值，以平滑绘图为目标，推荐选择 method='akima'。\n",
    "- 另外，method='akima'，method='barycentric' 和 method='pchip' 需要 Scipy 才能使用。"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "e9bad91a78804bec"
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 独热编码\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7ae01707e5ad9c5"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [],
   "metadata": {
    "collapsed": false
   },
   "id": "b3bd8b75046de139"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
